This should at least make use of acsStore.waitUntilAcsIngested.
It may be even better to wait if the round being looked up has not been ingested yet? we could compare max round in active table to assess that, and use acsStore.waitUntilRecordTimeReached?

If doing wait here is not possible, then it may be better to return something other than None?

If None is an acceptable response in such cases, (because the trigger would try again after a while?), then it should be made clear in comments.

The scenario would happen when scan has restarted and is catching up, The CalculateRewardsV2 would come from ScanStore which may be ahead of DbScanRewardsReferenceStore.
Currently it seems that the only thing the trigger would do when receiving None is retry?
it won't give up right?
so doing retry at the trigger level may be fine

Definitely add acsStore.waitUntilAcsIngested, the call might throw an exception otherwise ("using a storeId before it was initialized").

IMO it's fine to have ACS stores return data for the current state of the store (i.e., return None if the mining round has not been ingested yet), and let callers handle the fact that different stores might be lagging behind by arbitrary amounts of time.

As for triggers, after retrieveTasks() returns a task, the trigger will retry the task a few times, but if it keeps failing the trigger will eventually give up and retrieve the next task.

As long as you make sure retrieveTasks() is implemented such that it properly resumes from the persisted state, you should be fine.

I would still make sure we are not failing tasks just because some store is not caught up. Tasks that fail even after retries produce log warnings, and we alert on those in production. So either have retrieveTasks() only return tasks that have a reasonable chance of succeeding, or successfully complete a task if some data is missing.

Fixed: Thanks for all the feedback here, I've added waitUntilAcsIngested as recommended

Definitely add acsStore.waitUntilAcsIngested, the call might throw an exception otherwise ("using a storeId before it was initialized").

IMO it's fine to have ACS stores return data for the current state of the store (i.e., return None if the mining round has not been ingested yet), and let callers handle the fact that different stores might be lagging behind by arbitrary amounts of time.

As for triggers, after retrieveTasks() returns a task, the trigger will retry the task a few times, but if it keeps failing the trigger will eventually give up and retrieve the next task.

As long as you make sure retrieveTasks() is implemented such that it properly resumes from the persisted state, you should be fine.

I would still make sure we are not failing tasks just because some store is not caught up. Tasks that fail even after retries produce log warnings, and we alert on those in production. So either have retrieveTasks() only return tasks that have a reasonable chance of succeeding, or successfully complete a task if some data is missing.

I'm always nervous when we're creating indexes on non-empty tables in flyway migrations, because migrations are blocking application startup. AFAICT, this table should be sufficiently small that the index creation completes in a second, so it's fine.

Noted, thanks!